Augmented Reality, Computer Vision, and Digital Ticketing Systems

ABSTRACT

Augmented reality, computer vision, and digital ticketing system techniques are described that employ a location determination system. In one example, the location determination system is configured to receiving at least one digital image as part of a live camera feed, identify an object included in the at least one digital image using object recognition, determine a location of the object in relation to a digital map of a physical environment, generate augmented reality digital content indicating the determined location in relation to the digital map, and render the augmented reality digital content as part of the live camera feed for display by a display device.

RELATED APPLICATIONS

This application claims priority under 35 U.S.C. § 119(e) to U.S.Provisional Patent Application No. 62/619,071, filed Jan. 18, 2018, andtitled “Augmented Reality, Computer Vision, and Digital TicketingSystems,” the entire disclosure of which is hereby incorporated byreference.

BACKGROUND

Live events are generally organized and held in large venues that aredifficult to navigate. Individuals may find it challenging to accuratelyidentify the location of the venues that host events and require furtherassistance upon reaching the venues. Specifically, individuals may needassistance navigating through various areas within these venues in orderto find their seats, purchase food items and other merchandise, usevarious venue facilities, and find their way back to their seats.Additionally, users may need guidance in navigating through areas aroundthe venues, e.g. from the parking lot to the entrance etc.

Conventional tools and techniques that assist in such navigation areineffective because these tools and techniques are fractured across avariety of devices and applications. Additionally, these tools lacklocation determining functionalities that function effectively withinvenues due to lack of a signal or inadequate signal strength.Consequently, individuals' ability to comfortably attend and enjoy liveevents at large venues is frustrated and renders computing devices thatinclude these functionalities inoperable.

SUMMARY

Augmented reality, computer vision, and digital ticketing techniques aredescribed that are configured to facilitate user navigation within andaround physical environments via a computing device. A locationdetermination system, as implemented by the computing device describedherein, is configured to receive at least one digital image as part of alive camera feed of a physical environment. From this digital image, thelocation determination system determines a location with respect to adigital map of the physical environment. This may be performed in avariety of ways.

In a first example, text from an object captured in the digital image isrecognized, e.g., using text recognition and object recognition. Thetext, in this example is indicative of a location, e.g., a signindicating a corresponding section in a stadium, which is used todirectly determine a location with respect to a digital map. In a secondexample, the text alone is not indicative of a location, but is usableas part of a search to find a location, e.g., by recognizing a name of astore, concession stand, and so forth. The text in this second example,for instance, may identify the object but not the location. In a thirdexample, an object that does not include text is recognized, andsubsequently used as part of a search to determine the location, e.g.,objects of a concession stand indicating that a user is likelypositioned next to a stand, a statute at a sports stadium, and so forth.

From this, the location determination system determines a location ofthe object in relation to a digital map of a physical environment. Then,the system generates an augmented reality based digital content thatindicates the likely object location in relation to the digital map.This augmented reality based digital content assists users in traversingareas within the venue with ease, in part with the use of the identifiedlikely object location. Finally, the system renders the augmentedreality based digital content along with the live camera feed fordisplay by a display device of the computing device.

In this way, the location determination system described hereinovercomes the limitations of conventional tools and techniques thatassist in user navigation, namely the inability of these tools to assistusers in navigating from one location to another within venues. Incontrast with conventional tools and techniques, the system describedherein generates an augmented reality based digital content thatindicates the location of an object within the digital image, which thenserves as part of an internal venue guide for the user. In one example,the object identified in the digital image is an indicator of the user'slocation within the venue at a particular point in time. This locationis used in conjunction with other information associated with the user,e.g. user ticketing information, to guide the user in navigating toanother part of the venue, e.g. the user's seat. As a result, thelocation determination system described herein overcomes the limitationsof conventional tools and techniques of aiding user navigation.

This Summary introduces a selection of concepts in simplified form thatare further described below in the Detailed Description. As such, thisSummary is not intended to identify essential features of the claimedsubject matter, nor is it intended to be used as an aid in determiningthe scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. Entities represented in the figures may be indicative of one ormore entities and thus reference may be made interchangeably to singleor plural forms of the entities in the discussion.

FIG. 1 is an illustration of a location determination system operable toassist user navigation within a venue using augmented reality (AR)digital content that is generated by determining a location of an objectwithin the digital image.

FIG. 2 depicts, in greater detail, a system in an example implementationof the location determination system along with a content configurationmodule as part of the camera platform manager module of FIG. 1.

FIG. 3 depicts a system in an example implementation showing operationof the location determination system as employing object recognition ofan object identified in a digital image to determine a location.

FIG. 4 depicts an example of a physical environment as a stadium.

FIG. 5 depicts an example of text included in a digital image that isdirectly indicative of a location.

FIG. 6 depicts an example of text included in a digital image that isindirectly indicative of a location through identification of theobject.

FIG. 7 depicts an example of objects included in a digital image andobject recognition that does not include text that is indicative of alocation.

FIG. 8 is a flow diagram depicting a procedure in an exampleimplementation in which digital images are processed to determine alocation by the location determination system of FIG. 2.

FIG. 9 depicts an example implementation of the location determinationsystem of FIG. 2 that receives a digital image, a digital ticket, and 2Dand 3D maps, and generates AR digital content based on the receivedinformation.

FIGS. 10-28 depict examples of AR digital content rendered as part of alive feed of digital images based on the determined location of thelocation determination system.

FIG. 29 illustrates an example system including various components of anexample device that can be implemented as any type of computing deviceas described and/or utilized with reference to FIGS. 1-28 in order toimplement embodiments of the techniques described herein.

DETAILED DESCRIPTION

Overview

Conventional tools and techniques that assist user navigation within andaround physical environments encounter a variety of challenges.Individuals that purchase tickets to live events held at large venuesneed assistance in identifying the venue's location and navigating theareas within and around the venue. Specifically, individuals may desireassistance in identifying a fast route to their seats, in addition tofinding restrooms, merchandise shops, or concession stands near theseseats. Such navigational assistance improves the individuals' ability toenjoy attending live events. Conventional tools and techniques to do so,however, are ineffective in providing such assistance because thesetools may not be able to receive a signal or lack the requisite signalstrength within the venue to implement various location determinationfunctionalities. Moreover, conventional location determinationfunctionalities may be implemented using a variety of devices, makingthese tools and techniques cumbersome and inefficient.

Accordingly, a location determination system described herein addressesthese challenges and efficiently assists user navigation within venueswith the use of contextual information. Contextual information includeslocations of restrooms, signs, banners, concession stands, merchandiseshops, and other venue facilities. In one example of the locationdetermination system, the system initially receives at least one digitalimage as part of a live camera feed. The camera feed may include digitalcontent in the form of digital video or a collection of digital imageswithin a physical environment—inside the venue for instance.

The digital images may capture a variety of objects within the venue,e.g. signs, banners, items on the concession stands, and so forth. Fromthis, the system identifies at least one of the objects included in thedigital image via, for example, machine learning. After identifying theobject, the location determination system determines a location of theobject in relation to a digital map of a physical environment. Thelocation of the object within the venue serves as a marker or indicatorof the user's location (in relation to the identified object) at aparticular point in time, which may then be leveraged in a variety ofways, such as guiding a user to a desired location using a digital map.

The location within the venue may be identified using varioustechniques. In a first example, text associated with the identifiedobjects is identified using optical character recognition (OCR)techniques. In this example, the text may identify a current locationdirectly, e.g., indicate a section of the physical venue, seat location,and so forth.

In a second example, the text associated with the identified objects isusable to indirectly determine the objects' location by, e.g.,describing the object with which the text is associated. A digitalimage, for instance, may capture a sign of a merchandise store (theobject) with a banner containing the words “merchandise store.” Textindicating the name of the store may then be used as part of alookup/search to locate the store with respect to a digital map. In thisway, the location of the computing device used to capture the image isdetermined.

In a third example, objects that do not include text are recognizedusing object recognition, and subsequently used as part of a search todetermine a location. A computing device, for instance, may capture alive feed of digital images that include food items, drinks, bottles,and other items (i.e. objects) stacked on the shelves of a concessionstand. Identification of the object may be used to determine theobjects' location—on or near a concession stand. By consequence, theuser's location is also marked as being in or near the concession standwith respect to a digital map.

A computing device includes the location determination system may thenleverage the determined location in a variety of ways. After determiningthe location of the object in relation to a digital map of a physicalenvironment, for instance, the system generates augmented realitydigital content that indicates the determined object location inrelation to the digital image. Moreover, in one example, the augmentedreality based digital content is configured as a map that guides theuser from the location of the identified object to another location,e.g. the user's seat.

After determining the augmented reality digital content, the locationdetermination system described herein renders the augmented realitybased digital content along with the live camera feed for display by adisplay device of the computing device. The rendered augmented realitydigital content is accessible to the user via a mobile phone, handhelddevice, laptop, or other such device in real time. In this way, thelocation determination system described herein overcomes the limitationsof conventional techniques and facilitates effective user navigationwithin and around the areas of a venue such that the user's ability toattend and enjoy live events at these venues is improved. Furtherdiscussion of these and other examples is described in the followingdiscussion.

In the following discussion, an example environment is first describedthat may employ the techniques described herein. Example procedures andsystems are also described and shown as blocks which may be performed inthe example environment as well as other environments. Consequently,performance of the example procedures is not limited to the exampleenvironment and systems and the example environment and systems are notlimited to performance of the example procedures.

Example Environment

FIG. 1 is an illustration of a digital medium environment 100 in anexample implementation operable to employ computer vision and augmentedreality techniques described herein. The illustrated environment 100includes a computing device 102 that is communicatively coupled to aservice provider system 104 via a network 106. Computing devices thatimplement the computing device 102 and the service provider system 104may be configured in a variety of ways.

A computing device, for instance, may be configured as a desktopcomputer, a laptop computer, a mobile device (e.g., assuming a handheldconfiguration such as a tablet or mobile phone), configured to be worn(e.g., as goggles as illustrated for computing device 102) and so forth.Thus, a computing device may range from full resource devices withsubstantial memory and processor resources (e.g., personal computers,game consoles) to a low-resource device with limited memory and/orprocessing resources (e.g., mobile devices). Additionally, although asingle computing device is shown, a computing device may berepresentative of a plurality of different devices, such as multipleservers utilized by a business to perform operations “over the cloud”for the service provider system 104 as described in FIG. 29.

The computing device 102 is illustrated as being worn by a user in aphysical environment 108, e.g., a stadium or other large venue asillustrated whether indoor and/or outdoor. The physical environment 108in this example includes an event area and seating area 110. Thecomputing device 102 includes a digital camera 112 that is configured tocapture digital images 114 of the physical environment 108, such asthrough use of a charge coupled device (CCD) sensor. The captureddigital images 114 may then be stored as pixels in a computer-readablestorage medium and/or rendered for display by a display device, e.g.,LCD, OLED, LED, etc.

The computing device 102 also includes a camera platform manager module116 that is configured to implement and execute a camera platform 118(e.g., through use of a processing system and computer-readable storagemedia) that may serve as a basis for a variety of functionality. Thecamera platform 118, for instance, may implement a “live view” formed ofdigital images 114 taken of the physical environment 108 of thecomputing device 102. These digital images 114 may then serve as a basisto support other functionality.

An example of this functionality is illustrated as locationdetermination system 120. This may be implemented by the locationdetermination system 120 through use of the camera platform 118 in avariety of ways. In a first such example, the location determinationsystem 120 is configured to collect digital images 114. This may includedigital images 114 of physical objects in the physical environment 108in this example, a picture taken of a television screen or other displaydevice, and so on. The digital image 114 may also be captured of a userinterface output by the computing device 102, e.g., as a screenshot froma frame buffer.

The location determination system 120 includes object recognitionfunctionality to recognize objects included within the digital image114, e.g., via optical character recognition or machine learning. Fromthis, the location determination system 120 may collect data pertainingto the recognized objects. Data describing the recognized objects, forinstance, may be communicated via the network 106 to the serviceprovider system 104. The service provider system 104 includes a servicemanager module 122 that is configured to obtain data related to theobjects (e.g., through use of a search) from a storage device 124. Thisdata may then be communicated back to the computing device 102 via thenetwork 106 for use by the location determination system 120. In anotherexample, this functionality is implemented locally by the computingdevice 102.

The location determination system 120, for instance, may generateaugmented reality digital content 126 (illustrated as stored in astorage device 128) for output in the user interface of the computingdevice 102 as part of a “live feed” of digital images 114 taken of thephysical environment 106. The AR digital content 126, for instance, maydescribe a location of a seat, directions to the seat, a relation ofthat seat to other seats, directions to desired services available atthe physical environment 106, and so forth. This AR digital content 126is then displayed proximal to the object by the object inventory managermodule 120. In this way, the camera platform supports functionality forthe user 104 to “look around” the physical environment 106 and ascertainadditional information about objects included within the physicalenvironment.

In another example, the location determination system 120 leverages thecamera platform 118 to make recommendations for a user. The digitalimage 114, for instance, may also be processed using machine learning.In this example, the digital images are used to generate informationdescribing characteristics of the physical environment 108. Thesecharacteristics are then used as a basis to form recommendations (e.g.,through machine learning). Other examples are also contemplated, asfurther described in the following section.

In general, functionality, features, and concepts described in relationto the examples above and below may be employed in the context of theexample procedures described in this section. Further, functionality,features, and concepts described in relation to different figures andexamples in this document may be interchanged among one another and arenot limited to implementation in the context of a particular figure orprocedure. Moreover, blocks associated with different representativeprocedures and corresponding figures herein may be applied togetherand/or combined in different ways. Thus, individual functionality,features, and concepts described in relation to different exampleenvironments, devices, components, figures, and procedures herein may beused in any suitable combinations and are not limited to the particularcombinations represented by the enumerated examples in this description.

Location Determination System

FIG. 2 depicts a system 200 in an example implementation showingoperation of the location determination system 120 along with a contentconfiguration module as part of the camera platform manager module 116of FIG. 1 in greater detail. FIG. 3 depicts a system 300 in an exampleimplementation showing operation of the location determination system120 as employing object recognition of an object identified in a digitalimage 114 to determine a location. FIG. 4 depicts an example 400 of aphysical environment 108 as a stadium. FIG. 5 depicts an example 500 oftext included in a digital image that is directly indicative of alocation. FIG. 6 depicts an example 600 of text included in a digitalimage that is indirectly indicative of a location, e.g., throughidentification of the object itself. FIG. 7 depicts an example 700 ofobjects included in a digital image and object recognition that does notinclude text that, by itself, is indicative of a location. FIG. 8depicts a procedure 800 in an example implementation in which digitalimages are processed to determine a location of the computing device 102by the location determination system 120 of FIG. 2.

The following discussion describes techniques that may be implementedutilizing the previously described systems and devices. Aspects of theprocedure as shown stepwise may be implemented in hardware, firmware,software, or a combination thereof. The procedure is shown as a set ofblocks that specify operations performed by one or more devices and arenot necessarily limited to the orders shown for performing theoperations by the respective blocks. In portions of the followingdiscussion, reference will be made to FIGS. 2-8.

In this example, computer vision techniques are utilized to determine alocation of a computing device 102 with respect to a physicalenvironment 106, e.g., a sports venue. As shown in an example 400 ofFIG. 4, for instance, the physical environment 106 may include numerouscomplex levels that prove difficult for a user to navigate.Specifically, locating desired services, such as where an elevator 402is located may be challenging. Such information is useful because itenables users to locate restrooms 404, concessions 406, merchandise, andother services. An additional challenge may be the composition of thephysical environment 106 itself, which, due to the massive amounts ofconcrete and steel, inhibits receipt of signals that are used byconventional positional determination functionalities to ascertain aheight of a particular level. Resolving ambiguities associated with thisheight or addressing the challenges of calculating this height is alsoreferred to as a “z axis” problem. In the techniques described herein,however, the location determination system 120 leverages digital imagescaptured of the physical environment to determine a location, which maythen be used as a standalone functionality or to supplement and/orverify a location determined by other location determinationfunctionalities of the computing device 102, e.g., cell towertriangulation, GPS, and so forth.

To begin, at least one digital image 114 is received from a digitalcamera 112 by the camera platform manager module 116 as part of a livecamera feed (block 802). The digital image 114, for instance may capturea physical environment 108, in which, the computing device 102 isdisposed. From this digital image 114, a location determination system120 of the camera platform manager module 116 determines a location 202.This may be performed locally by the computing device 102 and thus isoperable in situations in which conventional location determinationfunctional may fail.

Examples of functionality of the location determination system 120 todetermine the location are represented by an object recognition system204 and a natural language processing system 206. The object recognitionsystem 204 is configured to recognize an object 302 in the digital image114, such as text 304 or to identify a type of object as an objectidentifier 306, e.g., a “lamp,” “paper cup,” “food,” “shirts,” and soforth. The object recognition system 204, for instance, may beconfigured to recognize text using optical character recognition (OCR),optical word recognition, intelligent character recognition (ICR),intelligent word recognition (IWR), and so forth. In this way, textincluded in the object 302 may be identified and used as a basis todetermine a location as further described below.

In another instance, the object recognition system 204 employs a machinelearning module 308 to implement machine learning techniques to identifythe object 302 that is not based, at least solely, on text through useof one or more models 310.

The machine learning module 308, for instance, may be configured toemploy the models 310 as a classifier usable to recognize the objectusing machine learning, e.g., neural networks, convolutional neuralnetworks, deep learning networks, structured vector machines, decisiontrees, and so forth. The models 310, for instance, may be trained usingtraining digital images that are tagged with correspondingidentifications of the objects and/or characteristics of the objects.

In an implementation, these training digital images and tags areobtained from a commerce service provider system. The training digitalimages are tagged by sellers using the system. As a result, contrary toconventional techniques, a multitude of accurately tagged trainingdigital images may be obtained with minimal computation and user cost.Although illustrated as implemented locally by the computing device 102,this functionality may also be implemented in whole or in part by aservice provider system 104 via the network 106. Thus, the objectrecognition system 204 may be used to detect an object 302 included inthe digital image 114 and/or characteristics of the object 302 includedin the image (block 804).

A location of the object 302 in relation to a digital map of a physicalenvironment is then determined (block 806) by the location determinationsystem 120. This determination may be performed in a variety of ways. Ina first example, the location is determined directly from textassociated with the identified object (block 808). As shown in anexample 500 of FIG. 5, for instance, a digital image 114 is displayed aspart of a live feed captured of a physical environment 108, in which,the computing device 102 is disposed. The digital image 114 includes anobject 502 having text. The text in this example is indicative,directly, of a location with respect to the physical environment 108,e.g., “Section 110.” This text, identified from the object 502, may thenbe used to determine the location 202 with respect to a digital map ofthe physical environment 108, e.g., as a search.

In a second example, the location is determined by the locationdetermination system 120 indirectly from text associated with theidentified object (block 810). As shown in the example 600 of FIG. 6, adigital image 114 is captured of a physical environment, which in thisinstance is an indoor mall. Objects including text in this example aresigns associated with respective stores. Thus, the text 304 identifiesan object 302 at the location but does not directly identify a locationof the object 302.

Accordingly, this text 304 may be passed to a location lookup module 312to determine a location of the object 302, and from this, determinationa location of computing device 102 in relation to the object 302. Thelocation lookup module 312, for instance, may use the text 304 as asearch of object names (e.g., which may be indexed using the objectnames) associated with the digital map to determine where the computingdevice 102 is located with respect to the objects on the map. Additionalimage processing and mapping techniques may also be employed to increaseaccuracy of this determination, e.g., triangulation from multipleidentified objects such as the stores in the illustrated example.

The location lookup module 312 may also employ natural languageunderstanding as implemented by a natural language processing system 206to determine the location 202 from the text 304. Natural-languageprocessing as implemented by the natural-language processing module 130is configured to understand the text 304 received from the objectrecognition system 204. The natural language processing system 206, forinstance, may employ a neural network to generate a representation of acommunication from the computing device 102, and from this, understand“what is being expressed” by the text 304. This is then used todetermine the location 202 based on what is being expressed.

Natural language understanding as implemented by the natural languageprocessing system 206 may then be used to understand what is beingexpressed by this language, and from this, infer a location of thecomputing device. An advertisement, for instance, may include informallanguage that does not identify the object, itself, but rather is usableto infer a location expressed by the text. The advertisement, forinstance, may include directions to a corresponding store at thephysical venue. From these directions, the natural language processingsystem 206 may determine a current location of the computing device 102with respect to the physical environment 108.

In this way, the location determination system 120 may leverage textthat directly indicates a location (e.g., a section, seat number),indirectly indicates a location through identification of an object(e.g., a name of a store), and even more indirectly through text thatdoes not identify the object nor the location. Non-textual techniquesobject recognition techniques may also be employed by the locationdetermination system 120 as described in the following example.

In a third example, the location is determined by the locationdetermination system 120 using object recognition that is not based ontext (block 812). A machine learning module 308, for instance, mayinclude a plurality of models 310 that are trained as classifiers todetermine a probability that a digital image includes a particularobject. As shown in an example 700 of FIG. 7, for instance, the digitalimage 114 includes objects including drinks, a counter, popcorn, and apopcorn popper. The models 310 in this example generates objectidentifiers 306 based on this recognition. From the object identifiers,the location lookup module 312 determines a location 202. Continuingwith the previous example, the object identifiers of the drinks,counter, popcorn, and so on may be used by the location lookup module312 to infer that the computing device 102 is located proximal to aconcession stand, and further that the concession stand sells popcorn.This knowledge may then be leveraged by the location lookup module 312to locate concessions stands in relation to the digital map, and fromthis, determine the location 202. As a result, the locationdetermination system 120 may leverage object recognition in a variety ofdifferent ways to determine a location.

In an implementation, verification functionality is also supported toincrease accuracy and confidence that a determined location 202 “iscorrect.” This may be performed, for instance, by continued monitoringof determined locations such that movement between the locations isplausible, e.g., does not “skip around” between levels, different sidesof the physical environment 108, and so forth.

Returning again to FIG. 2, the determined location 202 is output by thelocation determination system 120 and received as an input by a contentconfiguration module 208. The content configuration module 208 isrepresentative of functionality to configure and render digital contentbased on the determined location 202. In the illustrated example,augmented reality digital content 1206 is generated based on thedetermined location 202 in relation to the digital map (block 814) andis then rendered as part of the live camera feed for display in a userinterface 210 by a display device (block 816). This may be performed ina variety of ways.

FIG. 9 depicts a system 900 showing operation of the locationdetermination system 120 in greater detail as generating AR digitalcontent 126 based on a determined location 202 of FIG. 2. In this system900, access of the location determination system 120 to data that is notaccessible in conventional systems is shown. The location determinationsystem 120, for instance, this access includes 2D maps 902, 3D maps 904,and 360-degree views of the physical environment 108 including thestadium and surrounding areas. Conventionally, these items wereaccessible via different applications and devices and thus involvedmodal interaction.

The location determination system 120 also includes access to digitalimages 114 captured by the digital camera 112, e.g., as part of a “livestream.” Access to the digital ticket 208 is also permitted, which mayinclude functionality usable to permit user access to the physicalenvironment (e.g., a bar code, QR code), data describing where suchaccess is permitted (e.g., suite number, seat number, section number,level, parking spot, field access), and so forth. This is used by thelocation determination system 120 to generate AR digital content 126,which is not possible in conventional systems.

As shown in FIGS. 10-28, for instance, the AR digital content 126 may begenerated to appear on a defined surface (e.g., flat surface) that isviewable by the user along with a view of a physical environment of theuser, e.g., a direct view or recreated indirectly by a display device.In this way, the AR digital content 126 appears as disposed within anactual physical environment 108 of the user 104.

The digital camera 112, for instance, may be used to generate digitalimages 114 which are then examined by the camera platform manager module116 to identify landmarks (e.g., a flat plane), on which, the AR digitalcontent 126 is to be rendered to appear as if actually disposed in theuser's physical environment 108. Visual characteristics are then usedindicate correspondence of different sections between different viewsaccessible as part of the AR digital content 126, e.g., the 2D maps 902,3D maps 904, and 360-degree view 906. In this way, correlation ofdifferent parts of these rendered digital content may be readily andefficiency determined by the user, which is not possible in theconventional fractured techniques used to determine location.

A launch point, for instance, of the location determination system 120in the user interface may start with a rendering of a 2D map 902. A usermay then browser for tickets and see color coding of section or othervisual differentiation techniques. An option is selectable to then causeoutput of a 3D maps 904 as rendered as part of AR digital content 126.The location determination system 120, for instance, may render the ARdigital content 126 in response to detected motions of the user toappear to “fly over” different locations of the maps. This may be usedto locate a car park, the relationship of the car park to a ticketpickup location and associated event (e.g., a “pregame” or tailgateparty), and a relationship of the ticket pickup location and event tothe actual stadium and location within the stadium that is accessiblevia the digital ticket 908, e.g., a particular physical seat at whichrights are purchased to observe the event at a stadium. As illustrated,placement pins may be used to indicate particular locations,characteristics of those locations, and suggested physical navigationbetween the locations.

As part of this interaction, the location determination system 120 mayalso support output of digital marketing content. The digital marketingcontent, for instance, may advertise other events that are scheduled tooccur at the stadium, items available for purchase, and so forth. Thismay be performed as a standalone screen or as part of a live feedthrough configuration as AR digital content.

A ticket provider, for instance, may sell tickets that are scatteredabout a city, as opposed to just a ring around a stadium. Accordingly,markers may be used to indicate these locations, such as lots that willbe available to customers. Each of those markers may be configured as apin that appears to float in the air above the parking lot. Upondetection of a user input selecting the marker, the locationdetermination system 120 generates AR digital content 126, whichindicates a location of the lot, the name of the lot if it has a brandedname or a business name, and a link that is selectable to obtaindirections to the parking lot.

Markers are also positioned above the stadium itself to indicate gametime, the address and name of the place, and the teams that will beplaying there that day. Another positional marker for ticket pickupoutside a venue (e.g. a baseball stadium) and event locations aredisplayed. Additionally, information about public transportation optionsto and near the venue—metros or buses for instance—as well asinformation about using these transportation options may be provided.The AR digital content 126 may also include ticket purchase information,hours of venue operation, and so forth. In this way, the locationdetermination system 120 may configure AR digital content 126 toleverage additional data in a context of an event in order to facilitateticket purchases and enable effective user navigation within a venuethat holds the event.

In a first example, the location determination system 120 completes avenue map and venue area map in the location of a user 104, like atrain, home, office, it does not matter, e.g., on a flat surface. Visualcharacteristics such as color-coding, which indicates correspondencebetween maps, e.g., different ticketing sections are included in the ARdigital content 126. Also, a status of inventory (e.g., availabletickets) may be used to indicate tickets available for purchase to theuser in real-time based on a live dataset. Such information is alsorendered in the AR digital content 126. Therefore, users are providedup-to-date, up-to-the-minute information and data on ticketavailability.

In another example, once the user purchases a ticket or reserves aticket, the location determination system 120 is used to view agenerated 3D map based on the purchased ticket. Specifically, the 3D mapdepicts the section in which the user's seat is located and/or theuser's seat itself with a positional pin or indicator, which informs theusers of his seat location. That information can also be shared withother people and is helpful for the user upon arriving at the stadium.The system may also allow users to trigger a purchase flow by selectingsections or from within the AR digital content.

In a further example, interaction within different sections is supportedto trigger a virtual view mode. A user, for instance, may locate asection, tap on it, and then be brought into a 360-degree virtual viewthat includes the above features. In other words, a user may invoke thatview from within the AR view. As such, there are two different types of3D experiences such that one is nested inside the other. For instance, auser may start with a bird's eye view of a 3D map 204, then tap asection to cause output of a 360-degree view 206 that gives anappearance as if a user was actually sitting in that section. The usermay then zoom back out and see the whole stadium at one time.

In yet another example, location information is used to providecontextual help to the user in order to assist in navigation. This mayinclude assisting a user in locating a seat or other access as specifiedby the digital ticket 208. This may also be leveraged to providecontextual information, such as to suggest a bathroom, concessions, andother items closer to a person's seat, rather than based on a user'scurrent position. Such assistance may also include routes from a parkinglot, pre-events, and other ticket related information, all of whichserves to guide user navigation.

A user, for instance, may pull into a parking lot and open an app todetermine which way to walk to get to a seat the fastest, or where theclosest bathroom or hotdog stand is to his seat, the fastest way in andout of the venue, and so forth. In this way, knowledge of the digitalticket 208 and seat location is more persistent and allows users to usetheir camera lenses to get way-finding signage that helps user orientthemselves as they navigate to locations within the venue.

An additional example of social presence functionality is employed. If auser, for instance, bought a ticket for an event and someone else thatknown to that user has also bought a ticket to the same event, digitalticket 208 information may be leveraged, such as to locate each other'sseats. This includes as an overlay of AR digital content 126 as part ofa live camera feed. A user, for instance, may open an app, hold up thephone, and pan it across the stands to view a marker that indicateswhere a friend “is sitting.” This may also be used to leverage friendsand other contacts as part of social media to see “where your friends,if any, are at.”

For instance, if a first user has a ticket and a second user has aticket, social data may be shared to find a common meeting place basedon user preferences. And with facial recognition, the camera platformmanager module 116 may indicate when a person is detected in a crowd.The location determination system 120, for instance, may detect when aperson is within range of a particular location and then activate theobject recognition portion of the camera to identify this person.

In yet another example, digital images 114 are used as part of a livecamera feed and object recognition to determine positional locations.These markers improve the navigational accuracy in stadiums and othervenues, which are difficult to navigate using conventional positiondetermining functionalities due to lack of a signal. For example, objectrecognition of a sign or banner may be leveraged to indicate a user'sposition within a 3D model configured as AR digital content 126. Thismay also help resolve problems with determining position along a “z”axis.

Example System and Device

FIG. 29 illustrates an example system generally at 2900 that includes anexample computing device 2902 that is representative of one or morecomputing systems and/or devices that may implement the varioustechniques described herein. This is illustrated through inclusion ofthe camera platform manager module 116. The computing device 2902 maybe, for example, a server of a service provider, a device associatedwith a client (e.g., a client device), an on-chip system, and/or anyother suitable computing device or computing system.

The example computing device 2902 as illustrated includes a processingsystem 2904, one or more computer-readable media 2906, and one or moreI/O interface 2908 that are communicatively coupled, one to another.Although not shown, the computing device 2902 may further include asystem bus or other data and command transfer system that couples thevarious components, one to another. A system bus can include any one orcombination of different bus structures, such as a memory bus or memorycontroller, a peripheral bus, a universal serial bus, and/or a processoror local bus that utilizes any of a variety of bus architectures. Avariety of other examples are also contemplated, such as control anddata lines.

The processing system 2904 is representative of functionality to performone or more operations using hardware. Accordingly, the processingsystem 2904 is illustrated as including hardware element 2910 that maybe configured as processors, functional blocks, and so forth. This mayinclude implementation in hardware as an application specific integratedcircuit or other logic device formed using one or more semiconductors.The hardware elements 2910 are not limited by the materials from whichthey are formed, or the processing mechanisms employed therein. Forexample, processors may be comprised of semiconductor(s) and/ortransistors (e.g., electronic integrated circuits (ICs)). In such acontext, processor-executable instructions may beelectronically-executable instructions.

The computer-readable storage media 2906 is illustrated as includingmemory/storage 2912. The memory/storage 2912 represents memory/storagecapacity associated with one or more computer-readable media. Thememory/storage component 2912 may include volatile media (such asrandom-access memory (RAM)) and/or nonvolatile media (such as read onlymemory (ROM), Flash memory, optical disks, magnetic disks, and soforth). The memory/storage component 2912 may include fixed media (e.g.,RAM, ROM, a fixed hard drive, and so on) as well as removable media(e.g., Flash memory, a removable hard drive, an optical disc, and soforth). The computer-readable media 2906 may be configured in a varietyof other ways as further described below.

Input/output interface(s) 2908 are representative of functionality toallow a user to enter commands and information to computing device 2902,and also allow information to be presented to the user and/or othercomponents or devices using various input/output devices. Examples ofinput devices include a keyboard, a cursor control device (e.g., amouse), a microphone, a scanner, touch functionality (e.g., capacitiveor other sensors that are configured to detect physical touch), a camera(e.g., which may employ visible or non-visible wavelengths such asinfrared frequencies to recognize movement as gestures that do notinvolve touch), and so forth. Examples of output devices include adisplay device (e.g., a monitor or projector), speakers, a printer, anetwork card, tactile-response device, and so forth. Thus, the computingdevice 2902 may be configured in a variety of ways as further describedbelow to support user interaction.

Various techniques may be described herein in the general context ofsoftware, hardware elements, or program modules. Generally, such modulesinclude routines, programs, objects, elements, components, datastructures, and so forth that perform particular tasks or implementparticular abstract data types. The terms “module,” “functionality,” and“component” as used herein generally represent software, firmware,hardware, or a combination thereof. The features of the techniquesdescribed herein are platform-independent, meaning that the techniquesmay be implemented on a variety of commercial computing platforms havinga variety of processors.

An implementation of the described modules and techniques may be storedon or transmitted across some form of computer-readable media. Thecomputer-readable media may include a variety of media that may beaccessed by the computing device 2902. By way of example, and notlimitation, computer-readable media may include “computer-readablestorage media” and “computer-readable signal media.”

“Computer-readable storage media” may refer to media and/or devices thatenable persistent and/or non-transitory storage of information incontrast to mere signal transmission, carrier waves, or signals per se.Thus, computer-readable storage media refers to non-signal bearingmedia. The computer-readable storage media includes hardware such asvolatile and non-volatile, removable and non-removable media and/orstorage devices implemented in a method or technology suitable forstorage of information such as computer readable instructions, datastructures, program modules, logic elements/circuits, or other data.Examples of computer-readable storage media may include, but are notlimited to, RAM, ROM, EEPROM, flash memory or other memory technology,CD-ROM, digital versatile disks (DVD) or other optical storage, harddisks, magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, or other storage device, tangible media, orarticle of manufacture suitable to store the desired information andwhich may be accessed by a computer.

“Computer-readable signal media” may refer to a signal-bearing mediumthat is configured to transmit instructions to the hardware of thecomputing device 2902, such as via a network. Signal media typically mayembody computer readable instructions, data structures, program modules,or other data in a modulated data signal, such as carrier waves, datasignals, or other transport mechanism. Signal media also include anyinformation delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media include wired media such as awired network or direct-wired connection, and wireless media such asacoustic, RF, infrared, and other wireless media.

As previously described, hardware elements 2910 and computer-readablemedia 2906 are representative of modules, programmable device logicand/or fixed device logic implemented in a hardware form that may beemployed in some embodiments to implement at least some aspects of thetechniques described herein, such as to perform one or moreinstructions. Hardware may include components of an integrated circuitor on-chip system, an application-specific integrated circuit (ASIC), afield-programmable gate array (FPGA), a complex programmable logicdevice (CPLD), and other implementations in silicon or other hardware.In this context, hardware may operate as a processing device thatperforms program tasks defined by instructions and/or logic embodied bythe hardware as well as a hardware utilized to store instructions forexecution, e.g., the computer-readable storage media describedpreviously.

Combinations of the foregoing may also be employed to implement varioustechniques described herein. Accordingly, software, hardware, orexecutable modules may be implemented as one or more instructions and/orlogic embodied on some form of computer-readable storage media and/or byone or more hardware elements 2910. The computing device 2902 may beconfigured to implement particular instructions and/or functionscorresponding to the software and/or hardware modules. Accordingly,implementation of a module that is executable by the computing device2902 as software may be achieved at least partially in hardware, e.g.,through use of computer-readable storage media and/or hardware elements2910 of the processing system 2904. The instructions and/or functionsmay be executable/operable by one or more articles of manufacture (forexample, one or more computing devices 2902 and/or processing systems2904) to implement techniques, modules, and examples described herein.

The techniques described herein may be supported by variousconfigurations of the computing device 2902 and are not limited to thespecific examples of the techniques described herein. This functionalitymay also be implemented all or in part through use of a distributedsystem, such as over a “cloud” 2914 via a platform 2916 as describedbelow.

The cloud 2914 includes and/or is representative of a platform 2916 forresources 2918. The platform 2916 abstracts underlying functionality ofhardware (e.g., servers) and software resources of the cloud 2914. Theresources 2918 may include applications and/or data that can be utilizedwhile computer processing is executed on servers that are remote fromthe computing device 2902. Resources 2918 can also include servicesprovided over the Internet and/or through a subscriber network, such asa cellular or Wi-Fi network.

The platform 2916 may abstract resources and functions to connect thecomputing device 2902 with other computing devices. The platform 2916may also serve to abstract scaling of resources to provide acorresponding level of scale to encountered demand for the resources2918 that are implemented via the platform 2916. Accordingly, in aninterconnected device embodiment, implementation of functionalitydescribed herein may be distributed throughout the system 2900. Forexample, the functionality may be implemented in part on the computingdevice 2902 as well as via the platform 2916 that abstracts thefunctionality of the cloud 2914.

CONCLUSION

Although the invention has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the invention defined in the appended claims is not necessarilylimited to the specific features or acts described. Rather, the specificfeatures and acts are disclosed as example forms of implementing theclaimed invention.

What is claimed is:
 1. A method implemented by a computing device, themethod comprising: receiving, by the computing device, at least onedigital image as part of a live camera feed; identifying, by thecomputing device, an object included in the at least one digital imageusing object recognition; determining, by the computing device, alocation of the object in relation to a digital map of a physicalenvironment; generating, by the computing device, augmented realitydigital content indicating the determined location in relation to thedigital map; and rendering, by the computing device, the augmentedreality digital content as part of the live camera feed for display by adisplay device.
 2. The method as described in claim 1, wherein theidentifying is performed using a machine-learning model as a classifieras part of machine learning.
 3. The method as described in claim 2,wherein the determining is performed by using the identified object as asearch query of the digital map.
 4. The method as described in claim 1,wherein the identifying includes recognizing text associated with theobject included in the digital image and the determining is performedusing the identified text.
 5. The method as described in claim 4,wherein the object is a sign indicative of the location with respect tothe physical environment.
 6. The method as described in claim 4, whereinthe object is a sign indicative of a service available at the locationwith respect to the physical environment.
 7. The method as described inclaim 6, wherein the service offers products or goods for sale at thelocation within the physical environment.
 8. The method as described inclaim 1, further comprising obtaining ticket information of a userassociated with an event at the physical environment and configuring theAR digital content as a map from the location to a location at thephysical environment indicated by the ticket information.
 9. The methodas described in claim 8, further comprising obtaining ticket informationof another user associated with the event at the physical environmentand configuring the AR digital content as the map to also indicate alocation of the other user with respect to the physical environment. 10.A method implemented by a computing device, the method comprising:receiving, by the computing device, at least one digital image as partof a live camera feed; identifying, by the computing device, textassociated with an object included in the at least one digital imageusing object recognition; determining, by the computing device, alocation of the object in relation to a digital map of a physicalenvironment based on the text; generating, by the computing device,augmented reality digital content indicating the determined location inrelation to the digital map; and rendering, by the computing device, theaugmented reality digital content along with the live camera feed fordisplay by a display device.
 11. The method as described in claim 10,wherein the object is a sign indicative of the location with respect tothe physical environment.
 12. The method as described in claim 10,wherein the object is a sign indicative of a service available at thelocation with respect to the physical environment.
 13. The method asdescribed in claim 12, wherein the service offers products or goods forsale at the location within the physical environment.
 14. The method asdescribed in claim 10, wherein the determining is performed usingnatural language understanding as part of machine learning.
 15. Acomputing device comprising: a digital camera configured to generate alive stream of digital images; a display device; a processing system;and a computer-readable storage medium having instructions storedthereon that, responsive to execution by the processing system, causesthe processing system to perform operations comprising: identifying anobject included in the at least one digital image of the live stream ofdigital images using object recognition; determining a location of theobject in relation to a digital map of a physical environment;generating augmented reality digital content indicating the determinedlocation in relation to the digital map; and rendering the augmentedreality digital content as part of the live camera feed for display by adisplay device.
 16. The computing device as described in claim 15,wherein the identifying is performed using a machine-learning model as aclassifier as part of machine learning.
 17. The computing device asdescribed in claim 16, wherein the determining is performed by using theidentified object as a search query of the digital map.
 18. Thecomputing device as described in claim 15, wherein the identifyingincludes recognizing text associated with the object included in thedigital image and the determining is performed using the identifiedtext.
 19. The computing device as described in claim 18, wherein thetext identifies a location of the object.
 20. The computing device asdescribed in claim 18, wherein the text identifies a name of the object.